Approvod  for  publio  r*leu5«; 
llstrlbutlon  unlimited. 


TECHNICAL  REPORT  SERIES 

ON 

SIGNAL  PLUS  NOISE  MODELS 


.rriEJiyj 


^Department  of  Mathematics 
SUniversity  of  California 
[jjDavis,  California 


AiS  l-uaCK  OFFICE  OF  SCIENTIFIC  RKoEAECH  (AFo>‘ 
;.'CTIC“  OF  TRANSMITTAL  TO  DDC 
;i?.  tochEical  report  has  been  reviewed  and 
-’-■nrovcd  Tor  public  release  lAW  AFR  190-i*i 
Distribution  Is  unlimited. 

A.  D.  BLOSE 

Technical  Information  Officer 


( 


A Characterization  of  Convoluted 
Geometric  Distributions 

Francisco  J.  Samaniego 
Technical  Report  No.  1 


tMay  1977 

Department  of  Mathematics 
University  of  California 

I Davis,  California 


I nv 

i‘<’>vr.rr/o 

j'.iSfl.'ICA'IO.-r 

risi'JiBiiiiCN.'iv;:  ‘K. 

. -V  1 

II. 

A 

( . or- 

f ' , 

1 

p 1 

_ J 

1 

This  research  was  supported  in  part  by  the  .\ir  Force  Office  of  Scientific 
Research  under  Contract  AK) SR- 7 7 -3 180. 


2 


' SUMMARY 

; Let  X be  a nonnegative  integer-valued  random  variable  wiiose  distribution 

is  that  of  the  sum  of  a geometric  variable  Y with  parameter  0 and  a nonnegative 
integer-valued  random  variable  Z independent  of  Y whose  distribution  does  not 
depend  on  9.  The  variable  X is  said  to  have  a convoluted  geometric  distribution. 
The  class  of  all  such  distributions  are  here  characterized  by  a system  of 
differential  equations  satisfied  by  their  probability  mass  functions.  The 
characterization  is  shown  to  be  useful  in  maximum  likelihood  estimation  of 
the  parameter  9 when  the  distribution  of  Z is  known. 

I.  INTRODUCTION 

There  are  many  sources  in  nature  that  give  rise  to  discrete  data  in  the 
form  of  signals  plus  noise.  For  example,  data  obtained  by  a Geiger  counter 
may  be  viewed  as  sums  of  counts  due  to  the  presence  of  a radioactive  substance 
and  counts  due  to  noise  or  static.  Other  examples  in  which  discrete  signal 
plus  noise  models  appropriately  describe  random  phenomena  occur  in  the  use  of 
sonar  for  bottom  fishing  (see  Cushing  (1973),  for  example),  in  physiological 
processes  such  as  synaptic  tran.smission  of  neural  impulses  (see  Katz  (1966), 
Samaniego  (1976)),  and  in  single  server  queues  in  which  individuals  arriving 
for  seim/ice  can  be  classified  into  mutually  exclusive  categories  (see,  for 
example,  Slionick  (1970)).  In  situations  such  as  tliese  when  the  observed 
random  variable  presents  itself  as  the  sum  of  independent  discrete  variables, 
its  distribution  may  be  described  as  a discrete  signal  plus  noise  distribution. 
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A one  parameter  distribution  might  be  viewed  as  appropriate  when  the  noise 
variable  can  be  observed  alone,  but  the  signal,  whose  distribution  belongs 
to  a one  parameter  family,  can  be  observed  only  in  the  presence  of  noise.  The 
noise  distribution  may  be  estimated  with  high  precision  based  on  auxiliary  sampling 
on  the  noise  variable,  and  thus  might  be  assumed  equal  to  the  empirical  distribution. 

Let  Y be  a geometric  random  variable  with  probability  mass  function 
given  by 

P^(Y  = k)  = (1  -0)9^,  k = 0,1,2,... 

where  0 ^ (0,1).  If  Z is  a nonnegative  integer-valued  random  variable 

independent  of  Y,  and  X = Y +Z,  then  the  probability  mass  function  of  X is 

the  convolution  of  the  sequences  [P„(Y  = i)}  and  '_P(Z  = i)j.  Such  convoluted 

y 

geometric  distributions  arise  in  a familiar  life  testing  context.  Suppose 
the  lifetime  (in  hours)  of  an  item  on  test  is  distributed  according  to  an 
exponential  distribution  with  density 

f^(x)  = \e  X > 0 

where  X > 0.  The  num.ber  of  hours  of  life  of  the  item  has  a geometric  dis- 
tribution with  parameter  9 = e If  the  items  on  test  are  not  inspected 

hourly,  and  the  lifetime  is  recorded  as  the  number  of  hours  before  the  wear- 
out  is  observed,  then  the  recorded  lifetime  might  be  modelled  as  a convoluted 
geometric  variable.  For  example,  if  inspection  occurs  independently  with 
probability  p each  hour.  Then  the  recorded  lifetim.e  is  the  sum  of  two 

independent  geometric  variables.  If  the  recording  of  a wear-out  is  delayed 
one  hour  with  probability  p,  then  the  recorded  lifetime  is  the  sum  of 
independent  geometric  and  Bernoulli  variables. 


Estimation  problems  for  specific  signal  plus  noise  distributions  have 


been  examined  by  several  authors.  For  example,  Gaffey  (1959)  constructed 
a consistent  extimator  for  the  distribution  of  one  component  of  a 
continuous  signal  plus  noise  distribution.  Sclove  and  Van  Ryzin  (1969) 
derived  method  of  moments  estimators  for  a variety  of  multiparameter  signal 
plus  noise  distributions.  There  has  been  only  limited  success  in  maximum 
likelihood  estimation  for  such  m.odels  due  to  the  cumbersome  nature  of  the 
likelihood  function,  '.-diich,  for  discrete  signals,  consists  of  the  product 
of  (possibly  infinite)  sums  involving  the  probability  mass  function  of  the 
signal.  Samaniego  (1976)  examined  maximum  likelihood  estimation  for  one 
parameter  convoluted  Poisson  distributions,  solving  the  estimation  problem 
for  certain  families  of  such  distributions. 

In  this  paper,  the  family  of  convoluted  geometric  distributions  is 
characterized  by  a system  of  differential  equations  satisfied  by  their 
probability  mass  functions.  An  application  of  the  characterization  result 
to  maximum  likelihood  estimation  of  the  geometric  parameter  is  presented. 

II.  THE  CHAR/\CTERIZATION  RESULT 

A variety  of  characterizations  of  the  geometric  distribution,  or  more 
generally,  the  negative  binomial  distribution,  may  be  found  in  the  literature. 
Ferguson  (1965)  characterizes  the  geometric  distribution  in  terms  of  the 
independence  of  the  minimum  and  the  difference  of  two  independent  discrete 
variables.  The  negative  binomial  distribution  is  characterized  by  Katz(19^6") 
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via  a difference  equation  satisfied  by  its  probability  mass  function 
together  with  a moment  condition.  A characterization  of  the  negative 
binomial  distribution  by  systems  of  differential  equations  satisfied  by 
its  probability  mass  function  was  obtained  by  Boswell  and  Patil  (1973), 

This  latter  result  bears  a resemblance  to  the  characterization  established 
here,  but  does  not  reduce  to  a characterization  of  the  geometric 
distribution  alone  and  is  based  on  differential  equations  of  a substantially 
different  form.  Moreover,  their  result  hypothesizes  the  existence  of 
moments  of  all  orders,  a requirement  not  made  in  the  result  below. 


Theorem.  Let  X be  a nonnegative  integer- valued  random  variable  whose 
distribution  is  indexed  by  a parameter  9 € (0,1).  Then 


1-9 


Li  . 

E 9^’^  Pg(X  = n - j)  Yn,Y9 


(1) 


j=l 


if,  and  only  if,  the  distribution  of  X is  a convolution  of  the  geometric 
distribution  with  parameter  9 and  the  distribution  of  a nonnegative  integer 
valued  random  variable  which  does  not  depend  on  9, 


Proof.  Let  [p . , i = 0, 1 , . , . } be  a sequence  of  constants  such  that 

GO 

p.6[0,l]*fi  and  E p.  = 1,  Let  X be  a random  variable  whose  probability 
^ i=0  ^ 

mass  function  is 


P„(X  = n)  = E (1  -9)9’-p  . . 

^ ---.A  n-1 


i 

j 
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(2) 


- 6 - 


n . n 

S 9^  . + E i(l  - 0)0’'"  p . 

. n n-1  . , n-1 

1=0  1=1 


P-(X  = n)  n -in  . . 

= -^T:T-  Vi 

j=i  i=j 

P (X  = n)  n 

= - i .Q  ■ + Z 0^  "Pq(X  = n - j). 

j=l 

Thus,  the  probability  mass  function  of  a convoluted  geometric  distribution 

satisfies  (1).  Conversely,  suppose  the  distribution  of  X satisfies  (1). 

We  show  that  there  exists  a sequence  [p  ^ , i = 0 , 1 with  p.€  [0,1]  Yi, 

such  that  P.(X  = n)  satisfies  (2)  Yn.  The  fact  that  Z p.  = 1 follows  from 
y i 

the  equation  ZPg(X  = n)  = 1.  The  proof  is  by  induction.  For  n = 0,  we 


Pq(X  = 0) 


which  may  be  written 

+ <1^)2  '■/*=»)  = » 

Equating  the  indefinite  integrals  of  both  sides  of  (3),  we  obtain 

PgCX  = 0)  _ 


Pg(X  = 0)  = (1  -9)Pq. 

Since  (1  - 0)Pq  must  be  a probability  for  all  0 ^ (0,1),  we  have  that 
P-  € [0,1].  We  now  assume  that  for  r < k. 


P„(X  = r)  = Z (1  -9)  0 p^  . 
^ i=0 
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where  [0,1]  for  j = 0,1,. ..,r,  and  show  that  (2)  holds  for  n = k. 

We  have 

3 , Pf K = k j -1 

-P^(x.k) + p^(x-k-j) 

which,  by  the  induction  hypothesis,  may  be  written 


^ V!U1>  . ,1-1 , 

1-e  ..2  Pk-i  ’ 


(1  - 0)"  j=l  i=j 


which  simplifies  to 


Equating 


^P,(X-k)  Pg(X-k)  k 

1-0  2 i 6 Pk-i" 

(1  - 0)  i=l  ^ 

'"inite  integrals  of  both  sides  of  (4),  we  obtain 


(4) 


PQ(X=k)  k . 

* 1 - 0 ■ ® Pk-i  “^Pk* 

1=1 

The  fact  that  € [0,1]  follows  from  the  boundary  condition  that 

Pq(X  = k)  € [0,1]  for  all  0 € (0,1) . We  thus  have 

k 

P (X  = k)  = Z (1  -0)  0^  Pj^_. 
i=0 

with  p^  € [0,1]  for  j = 0,1,..,, k,  completing  the  proof. 


III.  AN  APPLICATION 


The  natural  domain  in  which  to  apply  the  characterization  result  above 
is  maximum  likelihood  estimation  of  the  geometric  parameter  0 under  the 
assumption  that  the  noise  distribution  {p^}  is  known.  As  stated  earlier, 
the  assumption  of  a known  noise  distribution  may  be  realized  when  the  noise 


variable  may  be  sampled  by  itself.  Maximum  likelihood  estimation  of  0 
may  be  facilitated  by  application  of  the  characterization  result  when  the 
monotonicity  property  below  obtains. 

Definition.  Let  fp  , 9 G ©]  be  a family  of  distributions  on  the  integers 
indexed  by  a real-valued  parameter  9.  The  family  is  said  to  have  strong  parametric 
monotone  decreasing  ratio  (strong  PMDR)  if  for  any  a < b receiving  positive 


mass  for  every  0 € 0,  the  ratio 


9^‘^'^Pq(X  = a) 

-Vx^bT- 


is  decreasing  in  9. 

This  definition  extends  the  notion  of  parametric  monotone  decreasing  ratio 
introduced  in  Samaniego  (1976)  to  a stronger  property  which  is  relevant  in  the 
estimation  problem  at  hand. 

Suppose  a random  sample  X^j,..,X^  is  obtained  from  a convoluted 

geometric  distribution.  Denote  the  likelihood  function  by 

L(x^,...,x^,0)  = n Pg(X.  = X.). 

1=1 

The  characterization  result  implies  that  the  likelihood  equation  An  L = 0 
may  be  written  as 

n Xi  9^’^  P (X  = X -j) 

E z — p (ir^n — = ° 

^ ® i=l  j=l  ^0^^i  "'i^ 


It  is  clear  from  (5)  that  if  a convoluted  geometric  distribution  has  strong 
PMDR,  the  derivative  of  the  likelihood  function  is  strictly  decreasing  and 
the  likelihood  equation  has  at  most  one  solution.  Thus,  the  maximum 
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likelihood  estimate  of  0 is  either  zero  or  the  unique  solution  of  the 
likelihood  equation.  The  MLE  may  be  approximated  numerically  to  any 
desired  degree  of  accuracy  using  the  fact  that  the  function  Zn  L may 
cross  zero  only  once. 

The  geometric  distribution  itself  has  strong  PMDR.  An  example  of  a convoluted 
geometric  distribution  with  strong  PMDR  is  given  below. 


Example  1.  Let  X = Y+Z  where  Y is  geometric  with  parameter  0 and  Z is 
Bernoulli  with  known  parameter  p.  Let  0 < a < b,  and  consider 


0^'^~^Pq(X  = a)  ^ 0*''^'^[a  - 0)0^'^P  + (1  - 0)0^(l-p)] 


Pq(X  = b) 


(1  - 9)0^‘^p  + (1  - 0)0^(l-p) 


(p  p + (1-p) 

P + 9 (1-p) 


which  is  decreasing  in  0.  If  a = 0 < b, 


9"~"''Pe(x  = 1-p 

Pg(X  = b)  P+0(1-P) 


which  is  also  decreasing  in  0.  Thus,  this  convoluted  geometric  distribution 
has  the  strong  PMDR  property. 

There  are  convoluted  geometric  distributions  for  which  the  maximum 
likelihood  estimation  of  0 poses  an  extremely  cumbersome  analytical  problem. 
Examples  can  be  constructed  in  which  the  likelihood  equation  has  any  fixed 
number  of  solutions.  The  example  below  illustrates  maximum  likelihood 


estimation  of  0 for  a convoluted  geometric  distribution  without  strong  PMDR. 
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Let  Z be  a random  variable  with  probability  mass  function 


P(2  = n) 


/^OOO 

2016 

200 

2016 

f420 
2016 

396 

2016 


if  n = 0 
if  n = 1 
if  n = 2 
if  n = 3 


otherwise 


Let  Y be  a geometric  variable  with  parameter  0,  and  let  X = Y + Z Suppose 
one  observation  is  taken  on  X,  and  X = 3 is  observed.  The  equat.' on  L = 0 
has  solutions  at  9 = .1,  .2  and  .3,  the  first  and  last  corresponding  to 
local  maxima.  The  likelihood  is  maximized  at  both  of  these  values,  and 
either  may  serve  as  the  MLE. 


Since  maximum  likelihood  estimation  of  9 is  in  general  equivalent  to 

n 

finding  zeros  of  a polynomial  of  degree  Z x.  , where  x is  the  vector  of 

i=l  ^ 

observations,  the  characterization  result  presented  here,  together  with  the 
strong  PMDR  property  when  it  is  applicable,  provide  a significant  simplific- 
ation of  the  problem. 
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